Non%2DNegative Matrix Factorization Clustering on Multiple Manifolds.

Predicting Correctness of Problem Solving in ITS with a Temporal Collaborative Filtering Approach.

Learning to Identify Students%27 Relevant and Irrelevant Questions in a Micro%2Dblogging Supported Classroom.

Discriminative models of integrating document evidence and document%2Dcandidate associations for expert search.

A joint probabilistic classification model for resource selection.

Probabilistic models for answer%2Dranking in multilingual question%2Danswering.

Combining evidence with a probabilistic framework for answer ranking and answer merging in question answering.

Discriminative graphical models for faculty homepage discovery.

Efficient privacy%2Dpreserving similar document detection.

Learning to Identify Students%27 Off%2DTask Behavior in Intelligent Tutoring Systems.

Learning from past queries for resource selection.

t%2DPlausibility Semantic Preserving Text Sanitization.
Text documents play significant roles in decision making and scientific research. Under federal regulations, documents (e.g., pathology records) containing personally identifiable information cannot be shared freely, unless properly sanitized. Generally speaking, document sanitization consists of finding and hiding personally identifiable information. The first task has received much attention from the research community, but the main strategy for the second task has been to simply remove personal identifiers. It is not hard to see that if important information (e.g., diagnoses and personal medical histories) is completely removed from pathology records, these records are no longer readable, and even worse, they no longer contain sufficient information for research purposes. Observe that the sensitive information “tuberculosis ” can be replaced with the less sensitive term “infectious disease”. That is, instead of simply removing sensitive terms, these terms can be hidden by more general but semantically related terms to protect sensitive information, without unnecessarily degrading the amount of information contained in the document. Based on this observation, the main contribution of this paper is to provide a novel information theoretic approach to text sanitization, develop efficient heuristics to sanitize text documents, and analyze possible attacks preventable under the proposed model. 1.
Predicting Correctness of Problem Solving from Low%2Dlevel Log Data in Intelligent Tutoring Systems.

Automatic Text Categorization of Mathematical Word Problems.

Multiple Instance Transfer Learning.
system performance diagnosis, machine learning, transfer learning, scalability Distributed systems continue to grow in scale and complexity, resulting in increasingly more involved interactions among components and increasingly more intricate failure modes that are very hard to diagnose manually. This increased vulnerability of larger systems, together with the increased difficulty of failure diagnosis, has motivated machine learning approaches to automate the diagnosis task. While preliminary encouraging results are achieved, scaling up the existing approaches to large applications remains challenging. With increase in scale, current approaches suffer the curse of dimensionality exacerbated by the exploding set of system states and measured metrics. In this paper, we significantly improve scalability of performance diagnosis methods. Our contributions lie in the use of (i) an intelligent partitioning of the metric space, coupled with a cooperative temporal segmentation algorithm, dividing system observations in time and in space to remove the multiplicative explosion of system states, and (ii) transfer learning techniques that improve accuracy by leveraging dependencies among the partitions. We validate our approaches on several months of production traces from a customer-facing geographically distributed, 24 x 7, 3-tier internet service. Our results show a significant accuracy improvement (35 % on average) over the naive partitioning of the state space (without the new temporal segmentation algorithm or transfer learning), and an order of magnitude reduction in computational cost over the “brute force ” approach of learning with no partitioning, without loss of accuracy.
M3IC Maximum Margin Multiple Instance Clustering.
Clustering, classification, and regression, are three major research topics in machine learning. So far, much work has been conducted in solving multiple instance classification and multiple instance regression problems, where supervised training patterns are given as bags and each bag consists of some instances. But the research on unsupervised multiple instance clustering is still limited. This paper formulates a novel Maximum Margin Multiple Instance Clustering (M 3 IC) problem for the multiple instance clustering task. To avoid solving a nonconvex optimization problem directly, M 3 IC is further relaxed, which enables an efficient optimization solution with a combination of Constrained Concave-Convex Procedure (CCCP) and the Cutting Plane method. Furthermore, this paper analyzes some important properties of the proposed method and the relationship between the proposed method and some other related ones. An extensive set of empirical results demonstrate the advantages of the proposed method against existing research for both effectiveness and efficiency. 1
Modeling search response time.

Similar Document Detection with Limited Information Disclosure.

Discriminative probabilistic models for passage based retrieval.
The approach of using passage-level evidence for document retrieval has shown mixed results when it is applied to a variety of test beds with different characteristics. One main reason of the inconsistent performance is that there exists no unified framework to model the evidence of individual passages within a document. This paper proposes two probabilistic models to formally model the evidence of a set of top ranked passages in a document. The first probabilistic model follows the retrieval criterion that a document is relevant if any passage in the document is relevant, and models each passage independently. The second probabilistic model goes a step further and incorporates the similarity correlations among the passages. Both models are trained in a discriminative manner. Furthermore, we present a combination approach to combine the ranked lists of document retrieval and passage-based retrieval. An extensive set of experiments have been conducted on four different TREC test beds to show the effectiveness of the proposed discriminative probabilistic models for passagebased retrieval. The proposed algorithms are compared with a state-of-the-art document retrieval algorithm and a language model approach for passage-based retrieval. Furthermore, our combined approach has been shown to provide better results than both document retrieval and passagebased retrieval approaches.
A Bayesian framework for knowledge driven regression model in micro%2Darray data analysis.

A Bayesian framework for knowledge driven regression model in micro%2Darray data analysis.

An effective and efficient results merging strategy for multilingual information retrieval in federated search environments.

Combining gene sequence similarity and textual information for gene function annotation in the literature.

A Probabilistic Framework for Answer Selection in Question Answering.
This paper describes a probabilistic answer selection framework for question answering. In contrast with previous work using individual resources such as ontologies and the Web to validate answer candidates, our work focuses on developing a unified framework that not only uses multiple resources for validating answer candidates, but also considers evidence of similarity among answer candidates in order to boost the ranking of the correct answer. This framework has been used to select answers from candidates generated by four different answer extraction methods. An extensive set of empirical results based on TREC factoid questions demonstrates the effectiveness of the unified framework. 1
A probabilistic graphical model for joint answer ranking in question answering.
Graphical models have been applied to various information retrieval and natural language processing tasks in the recent literature. In this paper, we apply a probabilistic graphical model for answer ranking in question answering. This model estimates the joint probability of correctness of all answer candidates, from which the probability of correctness of an individual candidate can be inferred. The joint prediction model can estimate both the correctness of individual answers as well as their correlations, which enables a list of accurate and comprehensive answers. This model was compared with a logistic regression model which directly estimates the probability of correctness of each individual answer candidate. An extensive set of empirical results based on TREC questions demonstrates the effectiveness of the joint model for answer ranking. Furthermore, we combine the joint model with the logistic regression model to improve the efficiency and accuracy of answer ranking.
Protecting source privacy in federated search.

Federated search of text search engines in uncooperative environments.

Combining Multiple Resources

Knowledge Transfer and Opinion Detection in the TREC 2006 Blog Track.
The paper describes the opinion detection system developed in Carnegie Mellon University for TREC 2006 Blog track. The system performed a two-stage process: passage retrieval and opinion detection. Due to lack of training data for the TREC Blog corpus, online opinion reviews provided in other domains, such as movie review and product review, were used as the training data. Knowledge transfer was performed to make the cross-domain learning possible. Logistic regression ranked the sentence-level opinions vs. objective statements. The evaluation shows that the algorithm is effective in the task.
A study of mixture models for collaborative filtering.

The FedLemur project Federated search in the real world.

Collaborative image retrieval via regularized metric learning.
In content-based image retrieval (CBIR), relevant images are identified based on their similarities to query images. Most CBIR algorithms are hindered by the semantic gap between the low-level image features used for computing image similarity and the high-level semantic concepts conveyed in images. One way to reduce the semantic gap is to utilize the log data of users ’ feedback that has been collected by CBIR systems in history, which is also called “collaborative image retrieval”. In this paper, we present a novel metric learning approach, named “regularized metric learning”, for collaborative image retrieval, which learns a distance metric by exploring the correlation between low-level image features and the log data of users ’ relevance judgments. Compared to the previous research, a regularization mechanism is used in our algorithm to effectively prevent overfitting. Meanwhile, we formulate the proposed learning algorithm into a semi-definite programming problem, which can be solved very efficiently by existing software packages and is scalable to the size of log data. An extensive set of experiments has been conducted to show that the new algorithm can substantially improve the retrieval accuracy of a baseline CBIR system using Euclidean distance metric, even with a modest amount of log data. The experiment also indicates that the new algorithm is more effective and more efficient than
CLEF 2005 Multilingual Retrieval by Combining Multiple Multilingual Ranked Lists.
Abstract: We participated in two tasks: Multi-8 two-years-on retrieval and Multi-8 results merging. For our multi-8 two-years-on retrieval work, simple multilingual ranked lists are first built by merging ranked lists of different languages that are generated by single types of retrieval algorithms. Then, algorithms are proposed to combine these simple multilingual ranked lists into a single ranked list. Empirical study shows that the approach of combining multilingual retrieval results can substantially improve the accuracies over single multilingual ranked lists. Multi-8 results merging task is our primary interest. This task is viewed as similar to the results merging task of federated search. Query-specific and language-specific models are proposed to calculate comparable document scores for a small amount of documents and estimate logistic models by using information of these documents. The logistic models are used to estimate comparable scores for all documents and thus the documents can be sorted into a final ranked list. A set of experiments demonstrated the advantage of the query-specific and language-specific models against several other alternatives.
Learn to weight terms in information retrieval using category information.

Adjusting Mixture Weights of Gaussian Mixture Model via Regularized Probabilistic Latent Semantic Analysis.
Abstract. Mixture models, such as Gaussian Mixture Model, have been widely used in many applications for modeling data. Gaussian mixture model (GMM) assumes that data points are generated from a set of Gaussian models with the same set of mixture weights. A natural extension of GMM is the probabilistic latent semantic analysis (PLSA) model, which assigns different mixture weights for each data point. Thus, PLSA is more flexible than the GMM method. However, as a tradeoff, PLSA usually suffers from the overfitting problem. In this paper, we propose a regularized probabilistic latent semantic analysis model (RPLSA), which can properly adjust the amount of model flexibility so that not only the training data can be fit well but also the model is robust to avoid the overfitting problem. We conduct empirical study for the application of speaker identification to show the effectiveness of the new model. The experiment results on the NIST speaker recognition dataset indicate that the RPLSA model outperforms both the GMM and PLSA models substantially. The principle of RPLSA of appropriately adjusting model flexibility can be naturally extended to other applications and other type of mixture models. 1
Modeling search engine effectiveness for federated search.
Federated search links multiple search engines into a single, virtual search system. Most prior research of federated search focused on selecting search engines that have the most relevant contents, but ignored the retrieval effectiveness of individual search engines. This omission can cause serious problems when federating search engines of different qualities.
Thresholding Strategies for Text Classifiers TREC 2005 Biomedical Triage Task Experiments.

York University at TREC 2005 Genomics Track.
Our Genomics experiments mainly focus on addressing three major problems in biomedical  information retrieval. The three problems are: (1) how to deal with synonyms? (2) how to deal  with the frequent use of acronyms? (3) how to deal with homonyms? In particular, we propose  two query expansion algorithms to construct structured queries for our experiments. The mean  average precision (MAP) for our automatic run &#034;york05ga1&#034; using Algorithm 1 was 0.2888 and  for our manual run &#034;york05gm1&#034; using Algorithm 2 was 0.3020. The evaluation results show  that both algorithms are e#ective for improving retrieval performance. We also find that some  other techniques such as pseudo-relevance feedback and using an extended stop word set can  make positive contributions to the retrieval performance.
Effective automatic image annotation via a coherent language model and active learning.
Image annotations allow users to access a large image database with textual queries. There have been several studies on automatic image annotation utilizing machine learning techniques, which automatically learn statistical models from annotated images and apply them to generate annotations for unseen images. One common problem shared by most previous learning approaches for automatic image annotation is that each annotated word is predicated for an image independently from other annotated words. In this paper, we proposed a coherent language model for automatic image annotation that takes into account the word-toword correlation by estimating a coherent language model for an image. This new approach has two important advantages: 1) it is able to automatically determine the annotation length to improve the accuracy of retrieval results, and 2) it can be used with active learning to significantly reduce the required number of annotated image examples. Empirical studies with Corel dataset are presented to show the effectiveness of the coherent language model for automatic image annotation. Categories and Subject Descriptors
Unified filtering by combining collaborative filtering and content%2Dbased filtering via mixture model and exponential model.

Unified utility maximization framework for resource selection.
This paper presents a unified utility framework for resource selection of distributed text information retrieval. This new framework shows an efficient and effective way to infer the probabilities of relevance of all the documents across the text databases. With the estimated relevance information, resource selection can be made by explicitly optimizing the goals of different applications. Specifically, when used for database recommendation, the selection is optimized for the goal of highrecall (include as many relevant documents as possible in the selected databases); when used for distributed document retrieval, the selection targets the high-precision goal (high precision in the final merged list of documents). This new model provides a more solid framework for distributed information retrieval. Empirical studies show that it is at least as effective as other state-of-the-art algorithms.
An automatic weighting scheme for collaborative filtering.
Collaborative filtering identifies information interest of a particular user based on the information provided by other similar users. The memory-based approaches for collaborative filtering (e.g., Pearson correlation coefficient approach) identify the similarity between two users by comparing their ratings on a set of items. In these approaches, different items are weighted either equally or by some predefined functions. The impact of rating discrepancies among different users has not been taken into consideration. For example, an item that is highly favored by most users should have a smaller impact on the user-similarity than an item for which different types of users tend to give different ratings. Even though simple weighting methods such as variance weighting try to address this problem, empirical studies have shown that they are ineffective in improving the performance of collaborative filtering. In this paper, we present an optimization algorithm to automatically compute the weights for different items based on their ratings from training users. More specifically, the new weighting scheme will create a clustered distribution for user vectors in the item space by bringing users of similar interests closer and separating users of different interests more distant. Empirical studies over two datasets have shown that our new weighting scheme substantially improves the performance of the Pearson correlation coefficient method for collaborative filtering.
Effect of varying number of documents in blind feedback analysis of the 2003 NRRC RIA workshop %22bf%5Fnumdocs%22 experiment suite.
INTRODUCTION  The &#034;bf_numdocs&#034; experiment was one of several sets of experiments conducted at the 2003 Reliable Information Access (RIA) Workshop, hosted by the Northeast Regional Research Center (NRRC) and MITRE, involving seven research groups. One of the goals for the RIA workshop was to focus on expansion techniques for relevance feedback and pseudo-relevance feedback. With this end in mind, the bf_numdocs suite of experiments was designed to examine the effect of varying the number of documents used to extract expansion terms for pseudo-relevance feedback.  We describe the motivation for this set of experiments, the hypotheses tested, an overview of the experimental method, and an analysis of results and conclusions. Given the nature of the workshop, we were able to examine whether there were any system-dependent effects or any topic-dependent effects.  2. HYPOTHESES  Several hypotheses were examined in the bf_numdocs experiment.  1. All the systems would demonstrate a tradeoff in
A study of methods for normalizing user ratings in collaborative filtering.
The goal of collaborative filtering is to make recommendations for a test user by utilizing the rating information of users who share interests similar to the test user. Because ratings are determined not only by user interests but also the rating habits of users, it is important to normalize ratings of different users to the same scale. In this paper, we compare two different normalization strategies for user ratings, namely the Gaussian normalization method and the decoupling normalization method. Particularly, we incorporated these two rating normalization methods into two collaborative filtering algorithms, and evaluated their effectiveness on the EachMovie dataset. The experiment results have shown that the decoupling method for rating normalization is more effective than the Gaussian normalization method in improving the performance of collaborative filtering algorithms.
A Bayesian Approach toward Active Learning for Collaborative Filtering.
Collaborative filtering is a useful technique for exploiting the preference patterns of a group of users to predict the utility of items for the active user. In general, the performance of collaborative filtering depends on the number of rated examples given by the active user. The more the number of rated examples given by the active user, the more accurate the predicted ratings will be. Active learning provides an effective way to acquire the most informative rated examples from active users. Previous work on active learning for collaborative filtering only considers the expected loss function based on the estimated model, which can be misleading when the estimated model is inaccurate. This paper takes one step further by taking into account of the posterior distribution of the estimated model, which results in more robust active learning algorithm. Empirical studies with datasets of movie ratings show that when the number of ratings from the active user is restricted to be small, active learning methods only based on the estimated model don’t perform well while the active learning method using the model distribution achieves substantially better performance. 1.
Collaborative filtering with decoupled models for preferences and ratings.
In this paper, we describe a new model for collaborative filtering. The motivation of this work comes from the fact that two users with very similar preferences on items may have very different rating schemes. For example, one user may tend to assign a higher rating to all items than another user. Unlike previous models of collaborative filtering, which determine the similarity between two users only based on their rating performance, our model treats the user&#039;s preferences on items separately from the user&#039;s rating scheme. More specifically, for each user, we build two separate models: a preference model capturing which items are favored by the user and a rating model capturing how the user would rate an item given the preference information. The similarity of two users is computed based on the underlying preference model, instead of the surface ratings. We compare the new model with several representative previous approaches on two data sets. Experiment results show that the new model outperforms all the previous approaches that are tested consistently on both data sets.
The Effect of Database Size Distribution on Resource Selection Algorithms.
Resource selection is an important topic in distributed information  retrieval research. It can be a component of a distributed information retrieval  task and can also serve as an independent application of database recommendation  system together with the resource representation part. There is a large body  of valuable prior research on resource selection but very little has studied about  the effects of different database size distributions on resource selection. In this  paper, we propose extended versions of two well-known resource selection algorithms:  CORI and KL divergence in order to consider the factors of database  size distributions, and compare them with the lately proposed Relevant Document  Distribution Estimation (ReDDE) resource selection algorithm. Experiments  were done on four testbeds with different characteristics, and the ReDDE  and the extended KL divergence resource selection algorithm have been shown  to be more robust in various environments.
Flexible Mixture Model for Collaborative Filtering.
This paper presents a flexible mixture model  (FMM) for collaborative filtering. FMM extends  existing partitioning/clustering algorithms for  collaborative filtering by clustering both users  and items together simultaneously without  assuming that each user and item should only  belong to a single cluster. Furthermore, with the  introduction of `preference&#039; nodes, the proposed  framework is able to explicitly model how users  rate items, which can vary dramatically, even  among the users with similar tastes on items.
Relevant document distribution estimation method for resource selection.
Prior research under a variety of conditions has shown the CORI algorithm to be one of the most effective resource selection algorithms, but the range of database sizes studied was not large. This paper shows that the CORI algorithm does not do well in environments with a mix of &#034;small&#034; and &#034;very large&#034; databases. A new resource selection algorithm is proposed that uses information about database sizes as well as database contents. We also show how to acquire database size estimates in uncooperative environments as an extension of the query-based sampling used to acquire resource descriptions. Experiments demonstrate that the database size estimates are more accurate for large databases than estimates produced by a competing method; the new resource ranking algorithm is always at least as effective as the CORI algorithm; and the new algorithm results in better document rankings than the CORI algorithm.
Preference%2Dbased Graphic Models for Collaborative Filtering.
Collaborative filtering is a very useful general technique for exploiting the preference patterns of a group of users to predict the utility of items to a particular user. Previous research has studied several probabilistic graphic models for collaborative filtering with promising results. However, while these models have succeeded in capturing the similarity among users and items, none of them has considered the fact that users with similar interests in items can have very different rating patterns; some users tend to assign a higher rating to all items than other users. In this paper, we propose and study two new graphic models that address the distinction between user preferences and ratings. In one model, called the decoupled model, we introduce two different variables to decouple a user’s preferences from his/her ratings. In the other, called the preference model, we model the orderings of items preferred by a user, rather than the user’s numerical ratings of items. Empirical study over two datasets of movie ratings shows that, due to its appropriate modeling of the distinction between user preferences and ratings, the proposed decoupled model significantly outperforms all the five existing approaches that we compared with. The preference model, however, performs much worse than the decoupled model, suggesting that while explicit modeling of the underlying user preferences is very important for collaborative filtering, we can not afford ignoring the rating information completely. 1.
A semisupervised learning method to merge search engine results.
This article presents a semisupervised learning solution to the result merging problem. The key contribution is the observation that information used to create resource descriptions for resource selection can also be used to create a centralized sample database to guide the normalization of document scores returned by different databases. At retrieval time, the query is sent to the selected databases, which return database-specific document scores, and to a centralized sample  database, which returns database-independent document scores. Documents that have both a database-specific score and a database-independent score serve as training data for learning to normalize the scores of other documents. An extensive set of experiments demonstrates that this method is more effective than the well-known CORI result-merging algorithm under a variety of conditions
A language modeling framework for resource selection and results merging.
Statistical language models have been proposed recently for several information retrieval tasks, including the resource selection task in distributed information retrieval. This paper extends the language modeling approach to integrate resource selection, ad-hoc searching, and merging of results from different text databases into a single probabilistic retrieval model. This new approach is designed primarily for Intranet environments, where it is reasonable to assume that resource providers are relatively homogeneous and can adopt the same kind of search engine. Experiments demonstrate that this new, integrated approach is at least as effective as the prior state-of-the-art in distributed IR.
Using sampled data and regression to merge search engine results.
This paper addresses the problem of merging results obtained from different databases and search engines in a distributed information retrieval environment. The prior research on this problem either assumed the exchange of statistics necessary for normalizing scores (cooperative solutions) or is heuristic. Both approaches have disadvantages.
Language model for IR using collection information.
early age of IR where documents are represented by the controlled vocabulary. In this paper, we explore the usage of meta-data information under the framework of language model. We present a new language model that is able to take advantage of the category information for documents to improve the retrieval accuracy. We compare the new language model with the traditional language model over the TREC4 dataset where the collection information for documents is obtained using the kmeans clustering method. The new language model outperforms the traditional language model, which verifies our statement.
A Statistical Model for Scientific Readability.
This paper presents a new method of using statistical models to estimate the reading difficulty of Web pages. Language Models are used to represent the content typically associated with different readability levels. Reading level classifiers are created as linear combinations of a language model and surface linguistic features. Experiments show that this new method is more accurate than the widely used Flesch-Kincaid readability formula  KEYWORDS  Readability, Flesch-Kincaid, Unigram Language Model, EM.  1. 
